4 research outputs found
Benchmarking Offline Reinforcement Learning on Real-Robot Hardware
Learning policies from previously recorded data is a promising direction for
real-world robotics tasks, as online learning is often infeasible. Dexterous
manipulation in particular remains an open problem in its general form. The
combination of offline reinforcement learning with large diverse datasets,
however, has the potential to lead to a breakthrough in this challenging domain
analogously to the rapid progress made in supervised learning in recent years.
To coordinate the efforts of the research community toward tackling this
problem, we propose a benchmark including: i) a large collection of data for
offline learning from a dexterous manipulation platform on two tasks, obtained
with capable RL agents trained in simulation; ii) the option to execute learned
policies on a real-world robotic system and a simulation for efficient
debugging. We evaluate prominent open-sourced offline reinforcement learning
algorithms on the datasets and provide a reproducible experimental setup for
offline reinforcement learning on real systems.Comment: The Eleventh International Conference on Learning Representations.
2022. Published at ICLR 2023. Datasets available at
https://github.com/rr-learning/trifinger_rl_dataset
Accelerated physical emulation of Bayesian inference in spiking neural networks
The massively parallel nature of biological information processing plays an
important role for its superiority to human-engineered computing devices. In
particular, it may hold the key to overcoming the von Neumann bottleneck that
limits contemporary computer architectures. Physical-model neuromorphic devices
seek to replicate not only this inherent parallelism, but also aspects of its
microscopic dynamics in analog circuits emulating neurons and synapses.
However, these machines require network models that are not only adept at
solving particular tasks, but that can also cope with the inherent
imperfections of analog substrates. We present a spiking network model that
performs Bayesian inference through sampling on the BrainScaleS neuromorphic
platform, where we use it for generative and discriminative computations on
visual data. By illustrating its functionality on this platform, we implicitly
demonstrate its robustness to various substrate-specific distortive effects, as
well as its accelerated capability for computation. These results showcase the
advantages of brain-inspired physical computation and provide important
building blocks for large-scale neuromorphic applications.Comment: This preprint has been published 2019 November 14. Please cite as:
Kungl A. F. et al. (2019) Accelerated Physical Emulation of Bayesian
Inference in Spiking Neural Networks. Front. Neurosci. 13:1201. doi:
10.3389/fnins.2019.0120
Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World
Experimentation on real robots is demanding in terms of time and costs. For
this reason, a large part of the reinforcement learning (RL) community uses
simulators to develop and benchmark algorithms. However, insights gained in
simulation do not necessarily translate to real robots, in particular for tasks
involving complex interactions with the environment. The Real Robot Challenge
2022 therefore served as a bridge between the RL and robotics communities by
allowing participants to experiment remotely with a real robot - as easily as
in simulation.
In the last years, offline reinforcement learning has matured into a
promising paradigm for learning from pre-collected datasets, alleviating the
reliance on expensive online interactions. We therefore asked the participants
to learn two dexterous manipulation tasks involving pushing, grasping, and
in-hand orientation from provided real-robot datasets. An extensive software
documentation and an initial stage based on a simulation of the real set-up
made the competition particularly accessible. By giving each team plenty of
access budget to evaluate their offline-learned policies on a cluster of seven
identical real TriFinger platforms, we organized an exciting competition for
machine learners and roboticists alike.
In this work we state the rules of the competition, present the methods used
by the winning teams and compare their results with a benchmark of
state-of-the-art offline RL algorithms on the challenge datasets